Variable operating mode HMD application management based upon crowd determined distraction

ABSTRACT

Disclosed is a system and method for managing variable operating mode applications that vary a cognitive load of the output of the applications presented to users wearing head mounted display devices. A tolerance level is indicative of the cognitive load of each operating mode of an application is established. In a vehicle setting, the operating mode of the application is selected to enable the safe operation of the vehicle based in part upon the tolerance level associated with the operating mode. If errant operations of vehicles are detected, then an application server modifies the tolerance level and synchronizes the modified tolerance level with head mounted display devices including the application.

BACKGROUND

The present description generally relates to the management of an outputof information of an application operating on a Head Mounted Displaysystem (HMD) based upon a crowd determined distraction level.

Advanced HMDs, including displays that resemble glasses, are becomingmore popular. While HMDs hold the potential advantage of enhancing thedriving experience, those concerned with safe vehicle operation considerthe value of banning the wearing of such devices while a user is drivinga vehicle because of the potential for driver distraction and resultingconsequences. Operating a vehicle requires certain cognitive load andmonitoring the output of an application requires a certain cognitiveload. If the cognitive load of the application is too great, unsafevehicle operation may result. However, a total ban on the use of HMDs ina vehicle may waste any potential advantages of HMDs.

SUMMARY

Disclosed is a method and apparatus for creating a safer environment forusers of HMDs in a moving vehicle. An application server monitors thedistraction level of users engaged in an activity, such as operating avehicle while wearing HMDs that are operating variable outputapplications. If the distraction level is unacceptable then a tolerancelevel of the application output is adjusted for all HMDs. The HMDcomprises a safety-checking module (SCM) that checks one or morereal-time attributes of the HMD, vehicle, and/or driver, and anaction-taking component (ATC) that takes a safety action, such asselecting a mode of the application, based on the SCM and the tolerancelevel of an application operating mode. Safety actions include reducingdistraction level and/or cognitive load of content conveyed to user ofHMD and/or changing car attributes such as reducing radio volume.

The disclosed subject matter makes use of HMD applications (Apps) withseveral modes associated with levels of distraction for a driver of avehicle. An App store is involved with the delivery of such Apps to theHMD, and maintains and delivers the Apps, in a useful way, to the driverwho uses an HMD. An automatic determination is made of user tolerance(T) for the levels (by crowdsourcing such as by learning through timevia vehicle information such as drifting off road, lane departures,sudden braking or other erroneous operations). A selection of theapplication mode is made based on T. A determination is made of a roleassociated with an HMD user in a car (e.g. driver vs. passenger). Basedon App mode and user role, an automatic determination is made withrespect to presentation for a multimodal HMD (e.g. HMD video mode, orHMD audio mode, or sending information to car audio). Benefits arepotentially realized by the “up-front” milieu associated with theservice (e.g. App store) that provides Apps for use on HMDs in cars. TheApp store maintains various information for future use and forassociation with various Apps and users, where the levels of distractionand associated App modes asserted by different App developers for theirapps are regularized by the service or store based on user feedback,measures from the vehicle, or crowdsourcing. The mode selection may beperformed in real time or may be determined by the App store prior to aparticular use context by a particular user.

In one example, a method comprises: at an application server, receivinga distraction signal based upon a user engaged in an activity whilewearing a remote head mounted display system presenting an output of anapplication having a plurality of operating modes for modifying anamount of information presented to the user wearing the remote headmounted display system, each of the plurality of operating modes havinga corresponding tolerance level for enabling the remote head mounteddisplay system to select one of the plurality of operating modes, thedistraction signal including a level of distraction of the user wearingthe remote head mounted display system during the activity and theoperating mode of the application; adjusting the tolerance level of atleast one of the plurality of operating modes based upon the distractionsignal; and enabling a second remote head mounted display system tooperate the application based upon the adjusted tolerance level.

In another example an application server comprises a distractionanalyzer operating a process adapted to: receive a distraction signalbased upon a user engaged in an activity while wearing a remote headmounted display system presenting an output of an application having aplurality of operating modes for modifying an amount of informationpresented to the user wearing the remote head mounted display system,each of the plurality of operating modes having a correspondingtolerance level for enabling the remote head mounted display system toselect one of the plurality of operating modes, the distraction signalincluding a level of distraction of the user wearing the remote headmounted display system during the activity and the operating mode of theapplication, and adjust the tolerance level of at least one of theplurality of operating modes based upon the distraction signal; and asynchronizer operating a process adapted to communicate the adjustedtolerance level to a second remote head mounted display system.

In another example, a computer program product comprises: a storagemedium readable by a processing circuit and storing instructions forexecution by the processing circuit of an application server forperforming a method comprising: receiving a distraction signal basedupon a user engaged in an activity while wearing a remote head mounteddisplay system presenting an output of an application having a pluralityof operating modes for modifying an amount of information presented tothe user wearing the remote head mounted display system, each of theplurality of operating modes having corresponding tolerance level forenabling the remote head mounted display system to select one of theplurality of operating modes, the distraction signal including a levelof distraction of the user wearing the remote head mounted displaysystem during the activity and the operating mode of the application;adjusting the tolerance level of at least one of the plurality ofoperating modes based upon the distraction signal; and enabling a secondremote head mounted display system to operate the application based uponthe adjusted tolerance level.

BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWINGS

The accompanying figures wherein reference numerals refer to identicalor functionally similar elements throughout the separate views, andwhich together with the detailed description below are incorporated inand form part of the specification, serve to further illustrate variousembodiments and to explain various principles and advantages all inaccordance with the present description, in which:

FIG. 1 illustrates an example block diagram of system for managingapplications having a variable operating mode for use with a remote headmounted system;

FIG. 2 shows an example flow diagram of a process enabling variableoperating mode application management for a remote head mounted displaysystem based upon crowd determined distractions in an applicationserver; and

FIG. 3 shows an example flow diagram of a process enabling variableoperating mode application management in a remote head mounted displaysystem based upon crowd determined distractions.

DETAILED DESCRIPTION

As required, detailed embodiments are disclosed herein; however, it isto be understood that the disclosed embodiments are merely examples andthat the systems and methods described below can be embodied in variousforms. Therefore, specific structural and functional details disclosedherein are not to be interpreted as limiting, but merely as a basis forthe claims and as a representative basis for teaching one skilled in theart to variously employ the present subject matter in virtually anyappropriately detailed structure and function. Further, the terms andphrases used herein are not intended to be limiting, but rather, toprovide an understandable description of the concepts.

The description of the present disclosure has been presented forpurposes of illustration and description, but is not intended to beexhaustive or limited in the form disclosed. Many modifications andvariations will be apparent to those of ordinary skill in the artwithout departing from the scope of the description. The embodiment waschosen and described in order to best explain the principles of thedescription and the practical application, and to enable others ofordinary skill in the art to understand the description for variousembodiments with various modifications as are suited to the particularuse contemplated.

HMDs come in many forms and form factors, and the scope of thisdescription is intended to cover a range of such wearable devices thatmay include one or two small displays with lenses, with semi-transparentmirrors embedded in a helmet, eye-glasses, visors, CRTs, LCDs,liquid-crystal on silicon (LCos), OLED, “curved Mirror” HMDs, and“waveguide” HMDs. Furthermore, other forms and form factors arepossible, such as displays embedded in contact lenses. The SCM(safety-checking module) may check any of the following: a detectionthat the HMD is on the face of the driver (e.g. a signal sent fromglasses, a camera viewing the driver, etc.), the nature and content ofthe information being conveyed to the user (e.g. driving directions,weather maps, etc.), maximum allowable cognitive load for the state inwhich the vehicle is currently being operated (e.g., state as determinedby GPS in vehicle or driver's smartphone), information relating to thecar (e.g. car speed, road conditions based on traction control, use ofbrakes), or information relating to the driver (e.g. history ofdriving). The system may also detect the need for taking a turn or exitin the next N seconds.

The HMD may also recognize a set of eye and facial gestures of thedriver to determine the alert level of the driver. For example, thegesture scanner may detect simple gestures such as yawning, eye closureor movements, etc. Methods for gesture detection and eye tracking areknown in the enabling art, and if such gestures are detected by the HMDsystem, the ATC (action-taking component) may take actions (as describedbelow to increase the probability of driver and pedestrian safety).Different information conveyed on the HMD (and different renderings ofinformation on the HMD) will present varying levels of distraction to adriver. For example, simply showing the temperature outside the car hasa low level of distraction. On the other hand, showing the driver acomplex animated weather map, along with the avatar of a talkingweather-person, provides a higher level of distraction. The ATC(action-taking component) may take various actions to increase theprobability of providing a safer driving experiences. For example, theATC may reduce distraction level and/or cognitive load of contentconveyed to user of HMD by simply showing the outside temperature ratherthan an animated avatar presenting a weather map.

Application providers may create different output modes for conveyinginformation so that a less-distracting rendition is presented when theneed for less distraction is automatically detected. These differentoutput modes may include alterations to the operating system (OS) of thedevice itself. For example a “HMD DRiver OS” may be provisioned by theATC when the user is driving, whereas a “HMD Full OS” is provisioned bythe ATC when the user is not driving. The provisioning of different OSversions may be automatic, as described above.

In another scenario, such as when driving in risky conditions, the ATCmay simply disable the HMD. Disabling a HMD that incorporatesprescription lenses while driving may be a statutory requirement if theHMD and the corrective lenses are inseparable. A user may realize thebenefit of corrective lenses and the HMD during their daily activities,while only benefiting from the corrective lenses while driving.

The ATC may trigger the taking of a vehicle safety action (e.g. reducesvolume of radio, at the permission of the user, based on a user profileor other kind of profile that allows such an option. The ATC may alsosend a signal to perform a real-time change to insurance fees orinsurance level.

The real-time distraction level and/or cognitive load provided by theHMD may be estimated by any of: creator of software (e.g. software thatdisplays information), crowdsourcing of users (e.g. votes) relating tosaid software, a real-time estimate of said load (e.g. car systemnotices the driver is driving erratically or the deterioration of roadconditions), analytics based on previous distraction level and/orcognitive load for similar drivers, information displays and roadconditions, etc.

A communicative coupling between a user's SCM/ATC and anotherindividual's device is also disclosed. For example, a parent orcaregiver may configure a ward's SCM such that it sends a signal whenused during driving. The parent or caregiver may then manually send areply to the ATC, causing it to take an action limiting the distractionlevel of the HMD on the driver. Note that in this example, this actionis applied to the HMD, not the car. For example, a parent may disablethe playing of videos on the HMD when a teenager is driving.

For certain scenarios, the SCM and/or ATC may be tailored to thespecific properties of a model of vehicle. For example, display featuresof the vehicle may be presented to a driver via his or her HMD, andtherefore this display will be under the control of the vehicle. Foruser specified functions and applications displayed on an HMD, the levelof safety and distraction produced, and thus the appropriate action onthe HMD that is required may also be functions of the vehicle, andtherefore may be optimally set by the vehicle manufacturer via thedesign of a vehicle's SCM and/or ATC. In this case, the SCM and ATCwould be part of the vehicle, and capable of accessing a driver's HMDvia standard interfaces to provide these safety functions.

In one example of operation, the SCM checks one or more real-timeattributes of vehicle (v), driver (d), and the information presentationon the HMD (h). The system computes a distraction level D based onD=f(h, v, d). Then the ATC takes a safety action based on D.

Note that although this description includes simultaneous use of HMDsand vehicle driving, it may have use in other scenarios such as whenwalking down a street filled with people, when operating heavymachinery, when operating a boat or plane, or when privacy is needed inoffices or meeting places.

In another example, an situational perimeter awareness component/moduleenables situational perimeter awareness (SPA) by scanning theenvironment in the vicinity of the vehicle. The SPA component functionsare triggered by analysis of situational conditions/attributes asdefined in by the SCM. The SCM component is constantly analyzingreal-time attributes and conditions coming from the HMD, vehicle, anddriver. When the SCM component determines that one or more SPA-definedconditions or attributes exist, the SPA may scan the defined perimeter.The SPA component uses sensors on the HMD and/or car to scan a definedperimeter for situational queues or objects. Situational queues orobjects in the scan perimeter may include other vehicles, pedestrians,pets, potholes, traffic lights, traffic signs, vehicle speed, rate ofacceleration/deceleration, or weather conditions. For example, thescanning component may scan perimeter ranges that could be set between45 and 360 degrees and up to 50 yards. Based on these features, the SPAthen analyzes the perimeter scan-data to identify any situational queuesor objects. Finally, the SPA interfaces in real-time with the SCM moduleto execute any corresponding ATC safety actions. In general, the purposeof the SPA is to facilitate safety actions that could greatly increaseor improve driver awareness and/or vehicle perimeter awareness, riskassessment, and safety-action response time.

The features and methods described in this disclosure involve a uniqueset of method components that can collect, analyze, advise, and executeboth human driver and vehicle safety practices in real time. Forexample, for the driver, the HMD component device delivers visual and/oraudio prompts that alert, inform, and instruct drivers real time onsituational safety and optional actions within a defined perimeter. Forthe integrated vehicle, the ATC component may optionally have aninterface with vehicle control systems to execute safety actions asdetermined by SCM analysis of key environment and situational datacollected by HMD and SPA module. Optionally, these vehicle controls maybe restricted to a small set of changes such as the reduction in radiovolume or the engaging of traction control, and this set of controls maybe approved by a vehicle operator before they are included as options toa system.

The terminology used herein is for the purpose of describing particularembodiments only and is not intended to be limiting of the description.As used herein, the singular forms “a”, “an” and “the” are intended toinclude the plural forms as well, unless the context clearly indicatesotherwise.

The terms “comprises” and/or “comprising,” specify the presence ofstated features, steps, operations, elements, and/or components, but donot preclude the presence or addition of one or more other features,integers, steps, operations, elements, components, and/or groupsthereof.

The term “application server” includes a device or system fordistributing and synchronization updates to software applications withremotely located devices operating the software. A remotely locateddevice includes a remote head mounted display systems coupled to theapplication server over a wired or wireless network such as theInternet, WiFi, or cellular networks.

The term “remote head mounted display system” includes a computer systemhaving a display that is worn on the head of a user and is remotelylocated from the application server. The computer system may beincorporated into the device worn on the head incorporating the displayor may be wired or wirelessly separated from the device worn on the headof the user. The remote head mounted display system may have a headsetoutput having one or two displays for presenting visual information toone or both eyes of the user, the headset output may also have one ortwo speaker systems for presenting audio information to one or both earsof the user. In other examples, the remote head mounted display systemmay have the display included in a structure that is not necessarilyworn by the user such as a cellphone, tablet, laptop or PC and furthermay be locally rather than remotely coupled to the application server.

The term “arrangement” means a system including an application serverand a remote head mounted display system.

The term “application” means a software module, process, or programdownloaded from the application server to the remote head mounteddisplay system for operation on the remote head mounted display system.An application of this description has a plurality of operating modesthat vary an amount of information presented to a user of a remote headmounted display system. The plurality of operating modes have associatedtolerance levels indicative of a cognitive load or the amount ofdistraction presented by the application to the user while operating inthat mode. The term application further includes an application suitecomprising a set of applications serving a similar purpose but withdifferent operating modes and different tolerance levels.

The term “activity” indicates a user involvement that is not necessarilyrelated to the comprehension of the output of the application. Inexample of an activity is the occupying a vehicle as either a driver ora passenger.

The term “vehicle” includes an automobile, motorcycle, truck, tractor,aircraft, watercraft, bicycle, skate board or other transportationdevice. The vehicle may have a telematics system that is able todetermine an error in operation of the vehicle, such as a lane departureor excessive braking, and generate an errant vehicle operation signal tothe application server either directly or through the remote headmounted display system. The vehicle may also have a vehicle output fordelivering audio and/or video information, including applicationinformation to the user.

The term “operating context” of a remote head mounted display systemmeans the environment in which the remote head mounted display system ispresenting information generated by the application to the user. Whilein a vehicle, the operating context includes a milieu signal whichincludes a role of the user, such as driver or passenger of a vehicle aswell as user age, user alertness, user driving history, and length oftime occupying vehicle. The operating context further includes a drivingcondition signal including an atmospheric condition, a road condition,or a traffic condition associated with the vehicle.

The term “distraction signal” indicates a signal indicative of a levelof distraction of the user wearing the remote head mounted displaysystem at the operating mode of the application.

FIG. 1 illustrates an example block diagram of system for managingapplications having a variable operating mode for use with a remote headmounted system. An application server 100 comprises a distractionanalyzer 102 adapted to receive a distraction signal based upon a user110 engaged in an activity such as driving a vehicle 112 while wearing aremote head mounted display system 115. The remote head mounted displaysystem is presenting an output of an application 120 having a pluralityof operating modes 122 for modifying an amount of information presentedto the user wearing the remote head mounted display system. Each of theplurality of operating modes has a corresponding tolerance level 124 forenabling the remote head mounted display system 115 to select one of theplurality of operating modes. The distraction signal includes a level ofdistraction of the user wearing the remote head mounted display systemduring the activity and the operating mode of the application. Thedistraction analyzer 102 further operates to adjust the tolerance level124 of at least one of the plurality of operating modes 122 based uponthe distraction signal. The application server 100 also includes asynchronizer 126 that operates a process to communicate the adjustedtolerance level to at least a second remote head mounted display system128.

The distraction server may receive a multiplicity of distraction signalsbased upon a crowd of a multiplicity of users engaged in the activitywhile wearing a corresponding multiplicity of remote head mounteddisplay systems 115, 128, 130, operating the application and adjust thetolerance level 124 of at least one of the plurality of operating modes122 based upon the multiplicity of distraction signals. The synchronizer126 further communicates the adjusted tolerance level to themultiplicity of remote head mounted display systems 115, 128, 130.

The remote head mounted display system 115 comprises an applicationmemory 140 for storing the application 120 downloaded from theapplication server 100. Applications are then operated on an applicationhost 141 which may include a microcomputer or other processor and anoutput which may be presented on a display. The remote head mounteddisplay system also includes a safety checking module 142 generating asafety check signal based upon an operating context of the remote headmounted display system 115 and a mode selector 144 selecting theapplication operating mode based upon the safety check signal and thetolerance level of the application operating mode. A distractiondetector 145 generates the distraction signal based upon a determinederror in an operation of the vehicle 112 within which the user 110 isoccupying and communicating the distraction signal to the applicationserver 100. Synchronizer 146 modifies the tolerance level of theapplication 120 stored in the application memory 140 based upon anadjusted tolerance signal received from synchronizer 126 of theapplication server 100.

In another example, safety signal can be communicated from the safetychecking module 142 to the application server 100 and the mode selectorprocess 144 included within the application server rather than theremote head mounted display system. A mode selection signal would thenbe communicated from the application server to the application host 141of the remote head mounted display system to operate the applicationmode based upon the adjusted tolerance level at the mode selected by theapplication server. In this example it not necessary to synchronize theapplication tolerance levels at both the application server and theremote head mounted display systems. Whether the mode selection processoccurs in the application server, or the remote head mounted displaysystem, the disclosure shows the remote head mounted display systemoperates the application based upon the adjusted tolerance level. In asimilar example, the regulation detector process 152 may be moved fromthe remote head mounted display system to the application server.

The remote head mounted display system 115 further comprises atelematics interface 148 for wirelessly interfacing with a telematicssystem 150 of the vehicle 112 for receiving an errant vehicle operationsignal based upon a vehicle determined error in an operation of thevehicle. An errant vehicle operation includes at least one of a lanedeparture, an excessive braking, and an excessive accelerationdetermination. In another example, the errant vehicle operation signalmay be sent directly from the vehicle telematics system 150 to theapplication server 100 without going through the remote head mounteddisplay system 115 using a cellular, WiFi or other coupling. In anotherexample, the errant vehicle operation signal may be generated by theremote head mounted display system 115. For example, excessive brakingmay be determined using a signal from an accelerometer included in theremote head mounted display system, and lane departures may be opticallydetermined using a camera system included in the head mounted displaysystem. Many other errant vehicle operations are possible, andconsidered to be within the scope of this description: such operationsinclude collisions, near collisions, excessive acceleration, improperturns, driving the wrong way on a one way street, running a stop sign ora stop light, exceeding speed limits, and failure to comply with othertraffic rules. In other examples the distraction signal may be receivedfrom any of a number of sources, for example from a survey about anoperating mode of the application completed by the user. The survey maybe offered in an on-line web page format soliciting responses from theuser for determining a level of distraction.

Safety checking module 142 further includes a regulation detector 152for further generating the safety check signal based upon a regulationssignal based upon changes in regulations governing a use of head mounteddisplay systems by users operating vehicles and communicated by aregulations compliance process 154 in the application server 100. Theregulations may be based upon local, state or federal government rulesas well as familial rules, established by parents of a child, employerrules established by employers of drivers of company vehicles of forthose driving on company business, or other rules established for otherpurposes. For example, if Nevada statutes required head mounted displayssystems worn while driving to present a reduced cognitive load to adriver relative to a passenger, and California statutes required anoutright ban of operating head mounted display systems operatingapplications while occupying a moving vehicle, then upon determining thelocation of the head mounted display system (using a global positioningsystem for example) the corresponding operation would be invoked. Inthis example, the head mounted display system would operate anapplication in mode having a reduced amount of information presented tothe user (a reduced cognitive load) while driving a vehicle in Nevada,and would disable the application while the user is occupying a movingvehicle in California.

The safety checking module 142 generates the safety check signal basedupon the milieu detector 154 which determines the role of the user,which in this example may determine if the user is a driver or apassenger in the vehicle 112 and/or a condition of the user including atleast one of user age, user alertness, user driving history, and lengthof time occupying vehicle. Safety checking module 142 also includes adriving condition detector 156 which detects driving conditions of thevehicle, such as a weather condition (rain, snow, ice, dark, etc.), aroad condition (free way, suburban, country, city, paved, dirt) and atraffic condition (rush hour, congested, lightly traveled).

The safety check modules uses the parameters to generate a safety signalindicative of the operating context of the remote head mounted displaysystem and thus the availability of the user to be exposed to an outputof an application operating on the remote head mounted display system.The application has a plurality of operating modes that modify theamount of information presented to the user (and thus vary cognitiveload presented by the application) as represented by the plurality oftolerance levels 124 associated with the plurality of operating modes122. The mode selector 144 matches the user's available cognitive loadwith the cognitive load presented by the application mode. Tolerancelevels 124 may derive from context of the remote head mounted displaysystem that may include a model of the dynamics of a user's cognitivestates, from which an estimate of available cognitive load may bederived.

For example, application 120 is a weather application and mode A has ahigh tolerance level of 8 and provides a larger amount of informationsuch as presenting an output at the remote head mounted display systemof a weather map with an avatar providing an audio description of theweather. Mode B has a medium tolerance level of 5 and presents audio andstatic visual icons regarding the current temperature and barometricconditions. Mode C has a low tolerance level of 2 and only presentsaudio information regarding the current temperature on the vehicle audiosystem. If the safety check module generates a safety signal generates avalue of 6 based on the context of the remote head mounted displaysystem, then operating mode B would be selected because the tolerancelevel of 5 (mode B) is less than 6, the value of the safety signal.

If the safety signal was calculated based on sunny dry drivingconditions, and if the conditions changed to dark and wet roads, thenthe value of the safety signal may be reduced from 6 to 4, and inresponse mode C selected because mode B's tolerance level of 6 exceedsthe value of the safety signal, while mode C has a tolerance level of 2which does not exceed the value of the safety signal. Thus, under sunnyand dry driving conditions the driver of the vehicle would experience aweather application that presents audio and static visual iconsregarding the current the temperature and barometric pressure conditionson the remote head mounted display system, while under dark and wetdriving conditions the driver would experience audio informationregarding the current temperature on the vehicle audio system.

Meanwhile, a rear seat passenger of the vehicle may experience theapplication in mode A, an output at the remote head mounted display of aweather map with an avatar providing an audio description of theweather, because a for rear seat passenger, the safety check signal mayproduce a value of 9 based upon the current context of the passenger'sremote head mounted display system having a low cognitive load, andtherefore allowing a greater additional load to experience theapplication. However, a front seat passenger of the vehicle mayexperience the application in mode A under sunny dry conditions and modeB under dark wet conditions in order to have additional cognitive loadavailable to assist the driver under difficult driving conditions.

The tolerance level of the modes of an application may be set by theapplication developer or other entity estimating the cognitive load ofthe mode of the application. However, this description goes further tomanage the tolerance level based upon inputs from various sourcesincluding the user of the application while performing variousactivities. The inputs may be received from multiple users, or a crowd.The tolerance level adjustment is done by monitoring erroneous behaviorof the user in the activity while the application is being operated. Forexample, a threshold level of lane departures per hour of driving may beacceptable for the previously mentioned weather application. Thethreshold may be 0.1% for example. The remote head mounted displaysystem, the vehicle telematics system, or both may report lanedepartures to the application server. Data from multiple, even thousandsof remote head mounted display systems may be gathered by theapplication server, and if the threshold is exceeded, then tolerancelevel may be decreased. For example, if the application serverdetermines that in mode B the application generated 0.2% lane departureswhen 0.1% is the threshold for the application, then the tolerance levelof mode B may be increased from 5 to 7. In the prior example, where thesafety checking signal had a value of 6, the application would nowoperate in mode C rather than mode B because of the excessive level oflane departures received from the application server monitoring thecrowd of users operating the application in mode B while driving. Inother examples, many more than three modes of an application may bedeveloped with a wider range of tolerance levels to allow theapplication operating mode to adapt in response to signals received fromthe crowd.

Also note that in one example, a first remote head mounted displaysystem operated by a first user of a first vehicle may generate a lanedeparture signal which causes the application server to modify thetolerance level of the application and communicate the modifiedtolerance level to another remote head mounted display system. Thus, thetolerance level of the mode of the application in the other remote headmounted display system is modified even though the user of the otherremote head mounted display system had not generated an erroneousoperation signal. Thus, the description provides the potential advantageof preemptively enhancing safety of the application of the other headmounted display system even though the other user had not yetencountered an erroneous operation. Furthermore, such preemptivemodification may be further based on a match between category of users'profiles, including such information as number of years driving, drivinghistory, and insurance level.

FIG. 2 shows an example flow diagram of a process enabling variableoperating mode application management for a remote head mounted displaysystem based upon crowd determined distractions in an applicationserver. Step 210 determines if a distraction signal has been receivedfrom a remote head mounted display system for an application and itscorresponding operating mode. If so, step 212 determines if the level ofdistraction has been exceeded and if so, the tolerance level is adjustedin response at step 214. The adjusted tolerance level is communicated atstep 216 resulting in the synchronization of the tolerance level of amode of operation of the application between the application server andat least one remote head mounted display system. Step 218 determines ifa change has been made to a statute or regulation governing theoperation of the remote head mounted display system, and if so aregulations signal is communicated to the remote head mounted displaysystem.

The process of FIG. 2 may be performed for applications operating on asingle remote head mounted display system, a plurality of remote headmounted display systems, or thousands or millions of remote head mounteddisplay systems while remaining within the scope of this description.The application server also conducts many other processes related to thedistribution of applications. Such other processes are known to thosefamiliar with the art.

FIG. 3 shows an example flow diagram of a process enabling variableoperating mode application management in a remote head mounted displaysystem based upon crowd determined distractions. The process may beimplemented upon non-transitory computer program product comprising astorage medium readable by a processing circuit and storing instructionsfor execution by the processing circuit. In the process, the remote headmounted display system downloads the application from the applicationserver at step 310. Step 312 determines if a tolerance level adjustmentand/or regulation signal is received from the application server. If so,the tolerance level and/or regulation signal associated with theapplication is adjusted, and is thereby synchronized with theapplication server at step 314. Thereafter, the operating context of theremote head mounted display system is determined and the safety checksignal is generated at step 318 and step 320 operates the application atthe selected operating mode. Step 322 then determines if an error hasoccurred in the vehicle operation while the application is operating,and if so, step 324 generates the distraction signal which iscommunicated to the application server along with the applicationoperating mode at step 326. Optional step 328 then reports intervalswhere the application is operating without a detected error in vehicleoperation. In other examples, the distraction signals and error freeoperation intervals may be stored in a batch and communicated toapplication server at convenient times, such as delaying until a WiFilink is available instead of using cellular bandwidth, or on a periodicbasis, such as hourly, daily, weekly or monthly.

FIG. 2 and FIG. 3 show that an application server has a process forreceiving a distraction signal based upon a user engaged in an activitywhile wearing a remote head mounted display system. The remote headmounted display system presents an output of an application having aplurality of operating modes. Each of the plurality of operating modesmodifies an amount of information presented to the user wearing theremote head mounted display system. Each of the plurality of operatingmodes has a corresponding tolerance level for enabling the remote headmounted display system to select one of the plurality of operatingmodes. The received distraction signal includes a level of distractionof the user wearing the remote head mounted display system during theactivity and the operating mode of the application. The applicationserver adjusts the tolerance level of at least one of the plurality ofoperating modes based upon the distraction signal and communicates theadjusted tolerance level to a second remote head mounted display system.The activity includes operating a vehicle and the level of distractionincludes an errant vehicle operation signal received from the vehicleand generated by a telematics system included within the vehicle. Thelevel of distraction includes an errant vehicle operation signal and isbased upon at least one of a detection of a lane departure of thevehicle and an excessive braking of the vehicle. In another example, thelevel of distraction includes an errant vehicle operation signal basedupon a determined error in an operation of the vehicle, as well as amilieu signal including a role of the user engaged in the activity, therole including at least one of a driver and a passenger of the vehicleand/or a driving condition signal based upon at least one of anatmospheric condition, a road condition, and a traffic conditionassociated with the vehicle occupied by the user.

FIG. 2 and FIG. 3 also show a remote head mounted display system fordownloading the application from the application server, generating asafety check signal based upon an operating context of the remote headmounted display system, selecting one of the plurality of operatingmodes of the application based upon the corresponding tolerance leveland the safety check signal, and operating the application at theselected operating mode.

In one example, selecting an operating mode may select anotheralternative application. For example, if a remote head mounted displayapplication had two weather applications one being a complex outputapplication requiring a significant cognitive load and without a modelike mode C of the aforementioned weather having a minimum tolerance of2, and the safety checking signal indicated that the minimum toleranceof an application should be two or less, then instead of loading themore complex weather application and not executing it because it doesnot have an operating mode with a tolerance level of 2 or less, theweather application of the aforementioned example would be loaded andoperated (because they are both weather applications) and the operatingmode selected would be C.

In another example, the activity includes occupying a vehicle and theremote head mounted display system includes a headset output forinterfacing with the user, and the vehicle includes a vehicle output forinterfacing with the user, wherein the selecting the applicationoperating mode includes directing the output of the application betweenthe headset output and the vehicle output. In this case if theaforementioned weather application was implemented and operating mode Bwas selected because the safety signal indicated a tolerance level of 5or less, then the output is sent to speakers only. However, if operatingmode A was selected because the safety signal indicated a tolerancelevel of 8 or less, then the output would be sent to speakers of thevehicle and display of the remote head mounted display system.

The process in the remote head mounted display system also includesdetermining an errant vehicle operation signal based upon a determinederror in an operation of the vehicle, generating the distraction signalbased upon the errant vehicle operation signal and the operating mode ofthe application, and communicating the distraction signal to theapplication server. The operating context of the remote head mounteddisplay system may be based upon at least one of a milieu signaldetermined based upon a role of the user, the role including at leastone of a driver and a passenger of the vehicle, and a driving conditionsignal determined based upon at least one of an atmospheric condition, aroad condition, and a traffic condition associated with the vehicle. Themilieu signal may further be determined based upon a condition of theuser, including at least one of user age, user alertness, user drivinghistory, and length of time occupying vehicle, and outputs of a usercognitive model.

The present invention may be a system, a method, and/or a computerprogram product. The computer program product may include a computerreadable storage medium (or media) having computer readable programinstructions thereon for causing a processor to carry out aspects of thepresent invention.

The computer readable storage medium can be a tangible device that canretain and store instructions for use by an instruction executiondevice. The computer readable storage medium may be, for example, but isnot limited to, an electronic storage device, a magnetic storage device,an optical storage device, an electromagnetic storage device, asemiconductor storage device, or any suitable combination of theforegoing. A non-exhaustive list of more specific examples of thecomputer readable storage medium includes the following: a portablecomputer diskette, a hard disk, a random access memory (RAM), aread-only memory (ROM), an erasable programmable read-only memory (EPROMor Flash memory), a static random access memory (SRAM), a portablecompact disc read-only memory (CD-ROM), a digital versatile disk (DVD),a memory stick, a floppy disk, a mechanically encoded device such aspunch-cards or raised structures in a groove having instructionsrecorded thereon, and any suitable combination of the foregoing. Acomputer readable storage medium, as used herein, is not to be construedas being transitory signals per se, such as radio waves or other freelypropagating electromagnetic waves, electromagnetic waves propagatingthrough a waveguide or other transmission media (e.g., light pulsespassing through a fiber-optic cable), or electrical signals transmittedthrough a wire.

Computer readable program instructions described herein can bedownloaded to respective computing/processing devices from a computerreadable storage medium or to an external computer or external storagedevice via a network, for example, the Internet, a local area network, awide area network and/or a wireless network. The network may comprisecopper transmission cables, optical transmission fibers, wirelesstransmission, routers, firewalls, switches, gateway computers and/oredge servers. A network adapter card or network interface in eachcomputing/processing device receives computer readable programinstructions from the network and forwards the computer readable programinstructions for storage in a computer readable storage medium withinthe respective computing/processing device.

Computer readable program instructions for carrying out operations ofthe present invention may be assembler instructions,instruction-set-architecture (ISA) instructions, machine instructions,machine dependent instructions, microcode, firmware instructions,state-setting data, or either source code or object code written in anycombination of one or more programming languages, including an objectoriented programming language such as Java, Smalltalk, C++ or the like,and conventional procedural programming languages, such as the “C”programming language or similar programming languages. The computerreadable program instructions may execute entirely on the user'scomputer, partly on the user's computer, as a stand-alone softwarepackage, partly on the user's computer and partly on a remote computeror entirely on the remote computer or server. In the latter scenario,the remote computer may be connected to the user's computer through anytype of network, including a local area network (LAN) or a wide areanetwork (WAN), or the connection may be made to an external computer(for example, through the Internet using an Internet Service Provider).In some embodiments, electronic circuitry including, for example,programmable logic circuitry, field-programmable gate arrays (FPGA), orprogrammable logic arrays (PLA) may execute the computer readableprogram instructions by utilizing state information of the computerreadable program instructions to personalize the electronic circuitry,in order to perform aspects of the present invention.

Aspects of the present invention are described herein with reference toflowchart illustrations and/or block diagrams of methods, apparatus(systems), and computer program products according to embodiments of theinvention. It will be understood that each block of the flowchartillustrations and/or block diagrams, and combinations of blocks in theflowchart illustrations and/or block diagrams, can be implemented bycomputer readable program instructions.

These computer readable program instructions may be provided to aprocessor of a general purpose computer, special purpose computer, orother programmable data processing apparatus to produce a machine, suchthat the instructions, which execute via the processor of the computeror other programmable data processing apparatus, create means forimplementing the functions/acts specified in the flowchart and/or blockdiagram block or blocks. These computer readable program instructionsmay also be stored in a computer readable storage medium that can directa computer, a programmable data processing apparatus, and/or otherdevices to function in a particular manner, such that the computerreadable storage medium having instructions stored therein comprises anarticle of manufacture including instructions which implement aspects ofthe function/act specified in the flowchart and/or block diagram blockor blocks.

The computer readable program instructions may also be loaded onto acomputer, other programmable data processing apparatus, or other deviceto cause a series of operational steps to be performed on the computer,other programmable apparatus or other device to produce a computerimplemented process, such that the instructions which execute on thecomputer, other programmable apparatus, or other device implement thefunctions/acts specified in the flowchart and/or block diagram block orblocks.

The flowchart and block diagrams in the Figures illustrate thearchitecture, functionality, and operation of possible implementationsof systems, methods, and computer program products according to variousembodiments of the present invention. In this regard, each block in theflowchart or block diagrams may represent a module, segment, or portionof instructions, which comprises one or more executable instructions forimplementing the specified logical function(s). In some alternativeimplementations, the functions noted in the block may occur out of theorder noted in the figures. For example, two blocks shown in successionmay, in fact, be executed substantially concurrently, or the blocks maysometimes be executed in the reverse order, depending upon thefunctionality involved. It will also be noted that each block of theblock diagrams and/or flowchart illustration, and combinations of blocksin the block diagrams and/or flowchart illustration, can be implementedby special purpose hardware-based systems that perform the specifiedfunctions or acts or carry out combinations of special purpose hardwareand computer instructions.

The description of the present application has been presented forpurposes of illustration and description, but is not intended to beexhaustive or limited to the description in the form disclosed. Manymodifications and variations will be apparent to those of ordinary skillin the art without departing from the scope of the description. Theexample was chosen and described in order to best explain the principlesof the description and the practical application, and to enable othersof ordinary skill in the art to understand the description for variousexamples with various modifications as are suited to the particular usecontemplated.

What is claimed is:
 1. A method comprising: at an application server,receiving a distraction signal based upon a user engaged in an activitywhile wearing a remote head mounted display system presenting an outputof an application having a plurality of operating modes for modifyinginformation presented to the user wearing the remote head mounteddisplay system, each of the plurality of operating modes having acorresponding tolerance level for enabling the remote head mounteddisplay system to select one of the plurality of operating modes, thedistraction signal including a level of distraction of the user wearingthe remote head mounted display system during the activity and theoperating mode of the application; adjusting the tolerance level of atleast one of the plurality of operating modes based upon the distractionsignal; and enabling a second remote head mounted display system tooperate the application based upon the adjusted tolerance level.
 2. Themethod according to claim 1 wherein the receiving receives amultiplicity of distraction signals based upon a multiplicity of usersengaged in the activity while wearing a corresponding multiplicity ofremote head mounted display systems operating the application, theadjusting adjusts the tolerance level of at least one of the pluralityof operating modes based upon the multiplicity of distraction signals,and the enabling further enables the multiplicity of remote head mounteddisplays system to operate the application based upon the adjustedtolerance level.
 3. The method according to claim 1 wherein the activityincludes operating a vehicle and the level of distraction includes anerrant vehicle operation signal received from the vehicle and generatedby a telematics system included within the vehicle.
 4. The methodaccording to claim 1 wherein the activity includes operating a vehicleand the level of distraction includes an errant vehicle operation signalis based upon at least one of a detection of a lane departure of thevehicle and an excessive braking of the vehicle.
 5. The method accordingto claim 1 wherein the activity includes occupying a vehicle and thelevel of distraction includes an errant vehicle operation signal basedupon a determined error in an operation of the vehicle, and a milieusignal including a role of the user engaged in the activity, the roleincluding at least one of a driver and a passenger of the vehicle. 6.The method according to claim 1 wherein the activity includes occupyinga vehicle and the level of distraction includes an errant vehicleoperation signal based upon a determined error in an operation of thevehicle, and a driving condition signal based upon at least one of anatmospheric condition, a road condition, and a traffic conditionassociated with the vehicle occupied by the user.
 7. The methodaccording to claim 1 further comprising: at the remote head mounteddisplay system downloading the application from the application server;generating a safety check signal based upon an operating context of theremote head mounted display system; selecting one of the plurality ofoperating modes of the application based upon the correspondingtolerance level and the safety check signal; and operating theapplication at the selected operating mode.
 8. The method according toclaim 7 wherein the selecting includes selecting an alternativeapplication.
 9. The method according to claim 7 wherein the activityincludes occupying a vehicle and the remote head mounted display systemincludes a headset output for interfacing with the user, and the vehicleincludes a vehicle output for interfacing with the user, wherein theselecting the application operating mode includes directing the outputof the application between the headset output and the vehicle output.10. The method according to claim 7 wherein the activity includesoccupying a vehicle and the method further comprises the steps of:determining an errant vehicle operation signal based upon a determinederror in an operation of the vehicle; generating the distraction signalbased upon the errant vehicle operation signal and the operating mode ofthe application; and communicating the distraction signal to theapplication server.
 11. The method according to claim 7 wherein theactivity includes occupying a vehicle and the operating context of theremote head mounted display system is based upon at least one of amilieu signal determined based upon a role of the user, the roleincluding at least one of a driver and a passenger of the vehicle, and adriving condition signal determined based upon at least one of anatmospheric condition, a road condition, and a traffic conditionassociated with the vehicle.
 12. The method according to claim 11wherein the milieu signal is further determined based upon a conditionof the user, the condition of the user including at least one of userage, user alertness, user driving history, and length of time occupyingvehicle.
 13. The method according to claim 7 further comprising: at theapplication server, generating a regulations signal based upon changesin regulations for a use of head mounted displays by users within anoperating context; and communicating the regulations signal to theremote head mounted display system, and at the head mounted display,receiving the regulations signal and the selecting further selects theone of the plurality of operating modes of the application based uponthe regulations signal.
 14. An application server comprising: adistraction analyzer operating a process adapted to receive adistraction signal based upon a user engaged in an activity whilewearing a remote head mounted display system presenting an output of anapplication having a plurality of operating modes for modifyinginformation presented to the user wearing the remote head mounteddisplay system, each of the plurality of operating modes having acorresponding tolerance level for enabling the remote head mounteddisplay system to select one of the plurality of operating modes, thedistraction signal including a level of distraction of the user wearingthe remote head mounted display system during the activity and theoperating mode of the application, and adjust the tolerance level of atleast one of the plurality of operating modes based upon the distractionsignal; and a synchronizer operating a process adapted to communicatethe adjusted tolerance level to a second remote head mounted displaysystem.
 15. The application server according to claim 14 wherein thedistraction analyzer further receives a multiplicity of distractionsignals based upon a multiplicity of users engaged in the activity whilewearing a corresponding multiplicity of remote head mounted displaysystems operating the application, and adjusts the tolerance level of atleast one of the plurality of operating modes based upon themultiplicity of distraction signals, and the synchronizer furthercommunicates the adjusted tolerance level to the multiplicity of remotehead mounted display systems.
 16. An application server according toclaim 14 further comprised with an arrangement including the remote headmounted display system, the remote head mounted display systemcomprising: an application memory storing the application; a safetychecking module generating a safety check signal based upon an operatingcontext of the remote head mounted display system; a mode selectorselecting the application operating mode based upon the safety checksignal and the tolerance level of the application operating mode; adistraction detector generating the distraction signal based upon adetermined error in an operation of a vehicle within which the user isoccupying and communicating the distraction signal to the applicationserver; and a synchronizer modifying the tolerance level based upon anadjusted tolerance signal received from the application server.
 17. Theapplication server according to claim 16 wherein the remote head mounteddisplay system further comprises a telematics interface for wirelesslyinterfacing with a telematics system of the vehicle for receiving anerrant vehicle operation signal based upon a vehicle determined error inan operation of the vehicle.
 18. The application server according toclaim 16 wherein the safety checking module further includes aregulation detector for further generating the safety check signal basedupon a regulations signal based upon changes in regulations governing ause of head mounted display systems by users operating vehicles andcommunicated by the application server to the remote head mounteddisplay system.
 19. A non-transitory computer program productcomprising: a storage medium readable by a processing circuit andstoring instructions for execution by the processing circuit of anapplication server for performing a method comprising: receiving adistraction signal based upon a user engaged in an activity whilewearing a remote head mounted display system presenting an output of anapplication having a plurality of operating modes for modifying anamount of information presented to the user wearing the remote headmounted display system, each of the plurality of operating modes havingcorresponding tolerance level for enabling the remote head mounteddisplay system to select one of the plurality of operating modes, thedistraction signal including a level of distraction of the user wearingthe remote head mounted display system during the activity and theoperating mode of the application; adjusting the tolerance level of atleast one of the plurality of operating modes based upon the distractionsignal; and enabling a second remote head mounted display system tooperate the application based upon the adjusted tolerance level.
 20. Thenon-transitory computer program product according to claim 19 whereinthe storage medium is readable by a second processing circuit andstoring instructions for execution by the processing circuit of a theremote head mounted display system for performing a method comprising:downloading the application from the application server; generating asafety check signal based upon an operating context of the remote headmounted display system; selecting one of the plurality of operatingmodes of the application based upon the corresponding tolerance leveland the safety check signal; and operating the application at theselected operating mode.